Autonomous vehicles and methods of using same

ABSTRACT

A vehicle includes one or more sensors arranged on at least one of a dashboard, a roof, and a center console of the vehicle, or one or more image capturing devices for capturing one or more images from a left side and a right side of the vehicle, an electronic control unit (ECU) configured to communicate with the one or more sensors or the one or more image capturing devices, and at least one of a morphing surface, a windshield display, and one or more displays configured to be controlled by the ECU, where the one or more sensors are arranged under a black panel surface.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a National Stage Entry of International PatentApplication No. PCT/EP2019/050451, filed Jan. 9, 2019, which claims thebenefit of priority to U.S. Provisional Patent Application No.62/615,249, filed Jan. 9, 2018, each of which is hereby incorporated byreference in its entirety for all purposes.

TECHNICAL FIELD

The present disclosure relates generally to various aspects ofautonomous driving vehicles and methods of using same. Such aspectsinclude sensor arrangements, overlay surfaces for sensors and displays,holographic displays, full windshield head up displays, transparentdashboards, touch pads, display cooling features, see through displays,arrangement of camera monitoring systems, systems and methods forcontrolling one or more vehicle components using one or more sensors,and systems and methods for controlling the display of one or moreimages using one or more image capturing devices.

BACKGROUND

The autonomous control of cars is increasingly becoming more common. Inthe limiting case, intervention by the driver in the driving operationis no longer necessary. In such a motor vehicle, it is no longernecessary for the driver of the motor vehicle to assume the usual,forward-facing sitting position. It is quite conceivable that in thefully autonomous operation of such a motor vehicle, the driver, forexample, may have a seat reclined in a rest position or, by rotation ofhis seat, occupies a position in which his view is directed to theoccupants in a back seat of the motor vehicle or towards a seatinggroup.

Even if direct intervention of the driver in the driving operation is nolonger necessary, it is still desirable in such a situation for thedriver or other occupants of the motor vehicle to obtain informationabout the current driving conditions. Moreover, in a sitting positionwhere the driver's gaze is no longer directed in the direction oftravel, there may be a problem with discomfort or nausea due tocontradictory information regarding the driver's visual perception andsense of balance.

Further, multiple new applications are developed for improvingexperience and safety in autonomous vehicles. Such developments mayinclude improvements to sensor arrangements, displays, dashboards, touchpads, cooling, and camera monitoring systems, among other aspects.

US 2016/311323 A1 teaches a vehicle display apparatus and a method forcontrolling the same. The display apparatus includes a display and acontroller configured to monitor a manipulation variable of a drivingmanipulation device provided to a vehicle, to select any one drivingmode corresponding to the manipulation variable of the drivingmanipulation device from among a plurality of predetermined drivingmodes, and to control the display to display different informationaccording to the plurality of driving modes. When a driving modecorresponding to a relatively small manipulation variable is selected,the controller decreases driving information displayed on the displayand increases convenience information displayed on the display, ascompared to when a driving mode corresponding to a relatively largemanipulation variable is selected.

US 2013/156296 A1 refers to a method and system for performing gesturerecognition of a vehicle occupant employing a time of flight (TOF)sensor and a computing system in a vehicle. An embodiment of the methodof the invention includes the steps of receiving one or more raw framesfrom the TOF sensor, performing clustering to locate one or more bodypart clusters of the vehicle occupant, calculating the location of thetip of the hand of the vehicle occupant, determining whether the handhas performed a dynamic or a static gesture, retrieving a commandcorresponding to one of the determined static or dynamic gestures, andexecuting the command.

US 2014/292665 A1 A refers to a control system, components andmethodologies that enable control of vehicle and in-vehicle systemfunctionality via gesture recognition. In illustrative embodiments, thecontrol system, constituent components and methodologies determine whatfunctionality a user wishes to control by tracking the user's gaze toidentify a selected function or function set. In combination withidentifying the selected function and function set, one or more sensorsinstalled in the vehicle detect one or more user gestures initiated by auser to interact with and/or control the selected function/function set.

US 2012/091744 A1 teaches a holding fixture operable to secure differingpluralities of positioned objects having differing geometric shapes,sizes, and/or dimensions, including an active material elementconfigured to selectively enable, cause, or retain the securingengagement and/or return the fixture to a ready state when not in use.

US 2010/066113 A1 refers to a center console comprising a manipulablestructural component, such as a pivotal lid, sliding armrest, tambourdoor, or pivotal cup holder, and at least one active material actuatorincluding an active material element operable to undergo a reversiblechange, drivenly coupled to the component, and configured toautonomously cause and/or enable the component to be manipulated as aresult of the change.

WO 2017/159863 A1 refers to an information processing device which isprovided with a detection unit, a display control unit, and adetermination unit. When the detection unit detects that the travelingdirection of a vehicle is rearward, the determination unit determinesthat a prescribed condition is met, the display control unit switchesfrom a composite image to a generated image and causes the generatedimage to be displayed by a display device.

EP 2 913 228 A1 teaches a passenger carrying vehicle comprising avehicle body and a passenger compartment at least partly surrounded bythe vehicle body, the vehicle body having an outer surface facing awayfrom the passenger compartment and an inner surface facing the passengercompartment. The vehicle body is arranged to prevent a passenger locatedin the passenger compartment from gaining a direct view of a vehiclepath in a primary direction of travel of the vehicle. The passengercompartment is thermally insulated and the vehicle comprises a sensorarrangement for gathering real-time data from the vehicle environmentand a display arrangement comprising a display device which is locatedin the passenger compartment. The display arrangement is arranged toreceive and display the real-time data from the sensor arrangement onthe display device.

SUMMARY

In one aspect a vehicle includes one or more sensors arranged on atleast one of a dashboard, a roof, and a center console of the vehicle,or one or more image capturing devices for capturing one or more imagesfrom a left side and a right side of the vehicle, an electronic controlunit (ECU) configured to communicate with the one or more sensors or theone or more image capturing devices, and at least one of a morphingsurface, a windshield display, and one or more displays configured to becontrolled by the ECU.

The vehicle may include the one or more sensors, where the ECU receivesdata from the one or more sensors on one or more of a position of anoccupant, a position of an object, a size of an object, or a shape of anobject.

The one or more sensors may be arranged under a black panel surface.

The one or more sensors may include at least one of a Time of Flight(ToF) sensor, a camera, an infrared (IR) sensor, a radar, an ultrasound,a capacitive sensor, a brightness sensor, and a LIDAR sensor.

The one or more sensors may include a plurality of sensors, and a firstsensor of the plurality of sensors may be configured to detect aposition of the occupant to activate or deactivate a second sensor or athird sensor of the plurality of sensors.

In response to the first sensor detecting that the position of theoccupant is outside a range of the second sensor and inside a range ofthe third sensor, the third sensor may be activated, and in response tothe first sensor detecting that the position of the occupant is outsidethe range of the third sensor and inside the range of the second sensor,the second sensor may be activated.

The second sensor and the third sensor may be positioned on a left sideand a right side of an instrument cluster display at a distance rangingfrom about 15 cm above to about 15 cm below the instrument clusterdisplay.

The one or more sensors may be configured to detect a position of atleast one of a center of a face of the occupant, an outer edge of a headof the occupant, a shoulder of the occupant, and a head of the occupantwith respect to a shoulder of the occupant.

Each of the one or more sensors may have a field of view of at leastabout 30 degrees and operates at a close range of at least about 5 cm.

The vehicle may include the windshield display, and content on thewindshield display may be configured to be moved depending on the datareceived from the one or more sensors.

The content on the windshield display may be configured to be moved tocompensate for movement of eyes of the occupant with respect to at leastone of an icon on the windshield display or an external object outsidethe vehicle.

The one or more sensors may be configured to detect one or more of anidentification, a drowsiness or fatigue, a distraction, a headorientation, an eye gaze, a facial expression, a gender classification,an age classification, a body type, a quantity, a hand gesture, a thumbsup gesture, an open palm gesture, a fist or fist gesture, a grabbing ofthe object, a releasing of the object, a proximity, and a proximity tothe object of or by the occupant.

The one or more sensors may be configured to detect one or more of atype of the object, a size of the object, an orientation of the object,and a position of the object.

The vehicle may further include one or more of an air vent, dashboardlighting, switches, a smartphone, a cup holder, a door pocket, a doorarmrest, a center console, a trunk, a seat, a seat back, and a roofwhich is configured to be controlled by the ECU.

The vehicle may include the morphing surface, and the morphing surfacemay be configured to change shape in response to a signal transmittedfrom the ECU, and the signal transmitted from the ECU may be initiatedusing the data received from the one or more sensors.

The morphing surface may include one or more electromechanical actuatorswhich are configured to change the shape of the morphing surface.

The ECU may be configured to initiate a self-test process to determinewhether the one or more sensors and the ECU are operating properly.

In response to detecting a malfunction of any of the one or more sensorsand the ECU, the ECU may be configured to display the detectedmalfunction or initialize a programmed malfunction protocol to self-curethe detected malfunction.

In response to no malfunction being detected, the ECU may be configuredto read an input of a first sensor of the one or more sensors until anarm movement is detected by the first sensor.

In response to an arm movement being detected by the first sensor, theECU may be configured to read a second sensor of the one or more sensorsto determine whether a recognized movement or gesture is performed, andin response to determining that a recognized movement or gesture isperformed, the ECU may be configured to transmit a signal to one or morevehicle components based on the recognized movement or gesture.

The one or more vehicle components may include the morphing surfacewhich is configured to change in shape based on the recognized movementor gesture.

A method of using the vehicle may include initiating a self-test processto determine whether the one or more sensors and the ECU are operatingproperly, displaying the detected malfunction or initialize a programmedmalfunction protocol to self-cure the detected malfunction in responseto detecting a malfunction of any of the one or more sensors and theECU, reading an input of a first sensor of the one or more sensors untilan arm movement is detected by the first sensor in response to nomalfunction being detected, reading a second sensor of the one or moresensors to determine whether a recognized movement or gesture isperformed in response to an arm movement being detected by the firstsensor, and transmitting a signal to one or more vehicle componentsbased on the recognized movement or gesture in response to determiningthat a recognized movement or gesture is performed.

The one or more vehicle components may include a morphing surface, andthe method may further include changing a shape of the morphing surfacebased on the recognized movement or gesture.

The vehicle may include the one or more image capturing devices forcapturing one or images from a left side and a right side of thevehicle, and the one or more displays, and the ECU may be configured tocontrol display of the one or more captured images on the one or moredisplays.

The one or more displays may include one or more of a camera monitoringsystem (CMS) dedicated displays, a center display, a dashboard display,a door display, and a rear view display.

A left side image captured by the one or more image capturing devicesmay be configured to be displayed on a first CMS dedicated display, aright side image captured by the one or more image capturing devices maybe configured to be displayed on a second CMS dedicated display, and theleft side image and the right side image may be configured to bestitched together and displayed on at least one of the center display,the dashboard display, and the door display.

The ECU may be configured to initiate a self-test process to determinewhether the one or more displays, the one or more image capturingdevices, and the ECU are operating properly.

In response to detecting a malfunction of any of the one or moredisplays, the one or more image capturing devices, and the ECU, the ECUmay be configured to display the detected malfunction or initialize aprogrammed malfunction protocol to self-cure the detected malfunction,

In response to no malfunction being detected, the ECU may be configuredto read or receive vehicle data to evaluate driving conditions.

In response to receiving vehicle data, the ECU may be configured todetermine whether to use a comfort display or a full display.

In response to determining to use the full display, the ECU may beconfigured to process the one or more captured images and display thefull display on the one or more displays.

In response to determining to use the comfort display, the ECU may beconfigured to process the one or more captured images and displaycomfort display on the one or more displays.

The ECU may be configured to determine whether to use the comfortdisplay or the full display using one or more of a vehicle speed, anumber of lanes on a road, a current driving lane, a time of driving, ablind spot detection, a vehicle acceleration, and a vehicle brakeactivation.

The ECU may be configured to determine to use the full display inresponse to a vehicle speed being less than 6 km/h.

The ECU may be configured to determine to use the comfort display inresponse to a vehicle speed being greater than or equal to 6 km/h, thevehicle speed may increase or decreases by 10 km/h for a duration of oneminute, only one lane being detected, a blind spot warning being off,and a vehicle brake being less than 0.1 g.

The ECU may be configured to determine to use the comfort display inresponse to a vehicle speed being greater than or equal to 6 km/h andmore than one lane being detected.

The ECU may be configured to determine to use the full display inresponse to a vehicle break being greater than 0.1 g.

The ECU may be configured to determine to use the full display inresponse to a blind spot warning being on.

The full display may include a separate display of a left side imagecaptured by the one or more image capturing devices and a right sideimage captured by the one or more image capturing devices, and thecomfort display may include a stitched image of the left side image andthe right side image on a single one of the one or more displays.

A method of using the vehicle may include initiating a self-test processto determine whether the one or more displays, the one or more imagecapturing devices, and the ECU are operating properly, displaying thedetected malfunction or initialize a programmed malfunction protocol toself-cure the detected malfunction in response to detecting amalfunction of any of the one or more displays, the one or more imagecapturing devices, and the ECU, reading or receiving vehicle data usingthe ECU to evaluate driving conditions in response to no malfunctionbeing detected, determining whether to use a comfort display or a fulldisplay using the ECU in response to receiving vehicle data, processingthe one or more captured images and displaying the full display on theone or more displays in response to determining to use the full display,and processing the one or more captured images and displaying thecomfort display on the one or more displays in response to determiningto use the comfort display.

The method may further include determining, using the ECU, whether touse the comfort display or the full display using one or more of avehicle speed, a number of lanes on a road, a current driving lane, atime of driving, a blind spot detection, a vehicle acceleration, and avehicle brake activation.

The method may further include determining to use the full display inresponse to a vehicle speed being less than 6 km/h.

The method may further include determining to use the comfort display inresponse to a vehicle speed being greater than or equal to 6 km/h, thevehicle speed increase or decreases by 10 km/h for a duration of oneminute, only one lane being detected, a blind spot warning being off,and a vehicle brake being less than 0.1 g.

The method may further include determining to use the comfort display inresponse to a vehicle speed being greater than or equal to 6 km/h andmore than one lane being detected.

The method may further include determining to use the full display inresponse to a vehicle break being greater than 0.1 g.

The method may further include determining to use the full display inresponse to a blind spot warning being on.

The displaying of the full display may include displaying a separatedisplay of a left side image captured by the one or more image capturingdevices and a right side image captured by the one or more imagecapturing devices, and the displaying of the comfort display may includedisplaying a stitched image of the left side image and the right sideimage on a single one of the one or more displays.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary, as well as the following detailed description,will be better understood when read in conjunction with the appendeddrawings. For the purpose of illustration, certain examples of thepresent disclosure are shown in the drawings. It should be understood,however, that the present disclosure is not limited to the precisearrangements and instrumentalities shown. The accompanying drawings,which are incorporated in and constitute a part of this specification,illustrate an implementation of system, apparatuses, and methodsconsistent with the present disclosure and, together with the detaileddescription, serve to explain advantages and principles consistent withthe present disclosure, wherein:

FIG. 1 is a side view of an exemplary arrangement of eye trackingsensors;

FIG. 2 is a side view of an exemplary arrangement of an upper Time ofFlight (ToF) sensor;

FIG. 3 is a side view of an exemplary arrangement of a center consoleToF sensor;

FIG. 4 is a cross-sectional view of an exemplary fastening structure;

FIGS. 5A and 5B are front perspective views of an exemplary displaydevice and seamless dashboard;

FIG. 6 is a diagram illustrating an exemplary Head Up Display (HUD)projector;

FIG. 7 is a rear perspective view of an exemplary projector model ofpico projectors;

FIG. 8 is a front perspective view of an exemplary projector model ofpico projectors;

FIG. 9 is a front perspective view of an exemplary attachment of a picoprojector in or at an A-pillar;

FIG. 10 is a perspective view of an exemplary cooling system;

FIG. 11 is a diagram illustrating an exemplary air guiding system;

FIG. 12 is a diagram illustrating an example of air flowing towards thedisplay and to the vent;

FIG. 13 is a side cross-sectional view of an exemplary armrest section;

FIG. 14 is a cross-sectional view of an exemplary basic structure of atouchskin configuration;

FIG. 15 is a flow chart illustrating an exemplary operable surface atdifferent stages;

FIG. 16 is a flow chart illustrating another exemplary operable surfaceat different stages;

FIG. 17 is a block diagram illustrating an exemplary see through displayfor parking purposes;

FIG. 18 is a front perspective view of an exemplary dashboard for anautonomous driving vehicle;

FIG. 19 is a diagram illustrating an exemplary system that can monitoror detect user movements or the environment for controlling a number ofvehicle components and conditions;

FIG. 20 is a flow chart illustrating an exemplary process of controllinga morphing surface using the one or more sensor arrangements andelectronic control unit;

FIG. 21 is a top plan view of an exemplary light guide on a vehicledashboard;

FIG. 22 is a perspective view of an exemplary vehicle door includingtouchskin keys;

FIG. 23 is a perspective view of an exemplary vehicle door includingsmart surfaces;

FIG. 24 is a perspective view of an exemplary see-through display anddisplay features;

FIG. 25 is a perspective view of an exemplary see-through display in avehicle door;

FIG. 26 is a front perspective view of an exemplary rear view display;and

FIG. 27 is a flow chart illustrating an exemplary process for displayingone or images on one or more displays.

DETAILED DESCRIPTION

It is to be understood that the disclosure is not limited in itsapplication to the details of construction and to the arrangements ofthe components set forth in the following description or illustrated inthe drawings. The Figures and written description are provided to teachthose skilled in the art to make and use the inventions for which patentprotection is sought. The disclosure is capable of other embodiments andof being practiced and carried out in various ways. Those skilled in theart will appreciate that not all features of a commercial embodiment areshown for the sake of clarity and understanding. Those skilled in theart will also appreciate that the development of an actual commercialembodiment incorporating aspects of the present disclosure may requirenumerous implementation-specific decisions to achieve the developer'sultimate goal for the commercial embodiment.

In addition, it is to be understood that the phraseology and terminologyemployed herein are for the purpose of describing the present disclosureand should not be regarded as limiting. For example, the use of asingular term, such as, “a” is not intended as limiting of the number ofitems. Also, the use of relational terms, such as but not limited to,“top,” “bottom,” “left,” “right,” “upper,” “lower,” “down,” “up,”“side,” are used in the description for clarity in specific reference tothe Figures and are not intended to limit the scope of the presentdisclosure. Further, it should be understood that any one of thefeatures may be used separately or in combination with other features.Other systems, methods, features, and advantages will be or becomeapparent to those skilled in the art upon examination of the Figures andthe description. The term “driver” is used throughout this disclosurebut is not limited to a person who is operating or controlling thevehicle; it may refer to any vehicle occupant, person, passenger, oruser inside the vehicle, or, in certain circumstances, a person who isoutside the vehicle but controlling the vehicle or interested inmovement of the vehicle. It is intended that all such additionalsystems, methods, features, and advantages be included within thisdescription, and be within the scope of the present disclosure.

Sensor Arrangements.

FIGS. 1-3 are side views of an exemplary arrangement of sensors 102,103, 104. FIG. 1 is a side view of an exemplary arrangement of eyetracking sensors 102, 103, 104.

Three eye tracking sensors 102, 103, 104 are attached to the dashboard110, each of the sensors 102, 103, 104 may include an IR lamp and acamera. In an example, the sensors 102, 103, 104 are mounted on thedashboard 110 as illustrated in FIG. 1. The distribution is arrangedoptimally using the field of view (hereinafter “FOV”) in order to alwaysrecognize the eyes of the driver 100. In this example, each sensor 102,103, 104 may be arranged under a black panel surface. A furtherdescription of the black panel surface is provided below under theheading Uniform Overlay Surface For Sensor And Display. In this example,the entire head of the driver 100 is always detectable. That is, inaddition to the eyes, the mouth, nose, ears, eye relief, etc. of thedriver 100 should also be detectable. Therefore, in one example, thesensors 102, 103, 104 should be positioned above the steering wheel, onthe upper edge of the dashboard 110, or on the front surface of thedashboard 110. While driving, one of the sensors 102, 103, 104 mayalways be active while the other sensors 102, 103, 104 are deactivated.Activation is determined by which sensor 102, 103, 104 can receivesufficient data about the vehicle occupant 100. Which sensor 102, 103,104 receives sufficient data about the vehicle occupant 100 isdetermined by where the occupant 100 looks. By determining the center ofthe face, another of the sensors 102, 103, 104 which corresponds withthe position of the face of the driver 100 is selected and activated.

For example, movement of the center of the face may be detected by afirst of the sensors 102, 103, 104 using parameters which indicatewhether the center of the face has moved out of the detection range ofan active sensor of the sensors 102, 103, 104. A number of differentdetecting processes may also be used to detect the position of the headand whether the head has moved outside the range of a sensor 102, 103,104. For example, the outer edge of the head or the axis of the head maybe detected/followed. Further, a position of the user's shoulder may bedetected/followed, and a position of the head with respect to theshoulders may be followed. Once the position is detected, the othersensor that can detect the face is activated. For example, one of thesensors 102, 103, 104 may be used to detect the position of the userwhile two of the sensors 102, 103, 104 may be activated or deactivateddepending on the detected position of the user in order to detect theuser's face. Any one of the sensors 102, 103, 104 may be a time offlight (hereinafter “ToF”) sensor or a camera.

In an example, each of the sensors 102, 103, 104 may have a 30° FOV, andwith the attachment of two of the sensors 102, 103, 104, a rotation ofthe head can be detected up to 100° in the horizontal direction. Inanother example, the FOV of each sensor may be at least 10°, at least20°, at least 30°, at least 40°, at least 50°, at least 60°, at least70°, at least 80°, at least 90°, at least 100°, at least 110°, at least120°, at least 130°, at least 140°, at least 150°, at most 10°, at most20°, at most 30°, at most 40°, at most 50°, at most 60°, at most 70°, atmost 80°, at most 90°, at most 100°, at most 110°, at most 120°, at most130°, at most 140°, at most 150°, about 10°, about 20°, about 30°, about40°, about 50°, about 60°, about 70°, about 80°, about 90°, about 100°,about 110°, about 120°, about 130°, about 140°, or about 150°.Additionally, the rotation of the head of the driver 100 may be detectedin the horizontal direction at least 20°, at least 30°, at least 40°, atleast 50°, at least 60°, at least 70°, at least 80°, at least 90°, atleast 100°, at least 110°, at least 120°, at least 130°, at least 140°,at least 150°, at least 160°, at most 20°, at most 30°, at most 40°, atmost 50°, at most 60°, at most 70°, at most 80°, at most 90°, at most100°, at most 110°, at most 120°, at most 130°, at most 140°, at most150°, at most 160°, about 20°, about 30°, about 40°, about 50°, about60°, about 70°, about 80°, about 90°, about 100°, about 110°, about120°, about 130°, about 140°, about 150°, or about 160°. In one example,the distance between the two of the sensors 102, 103, 104 may beapproximately 0.5 m. In another example, the distance between the twosensors may be at least 0.1 m, at least 0.2 m, at least 0.3 m, at least0.4 m, at least 0.5 m, at least 0.6 m, at least 0.7 m, at least 0.8 m,at least 0.9 m, at least 1 m, at most 0.1 m, at most 0.2 m, at most 0.3m, at most 0.4 m, at most 0.5 m, at most 0.6 m, at most 0.7 m, at most0.8 m, at most 0.9 m, at most 1 m, about 0.1 m, about 0.2 m, about 0.3m, about 0.4 m, about 0.5 m, about 0.6 m, about 0.7 m, about 0.8 m,about 0.9 m, or about 1 m. The distance of the occupant 100 to each ofthe sensors 102, 103, 104 may range and can be up to one meter, theminimum distance differs from person to person, but may be about 10 cmor else the sensor 102, 103, 104 may become saturated.

Still referring to FIG. 1, a model for an eye tracking sensor 102, 103,104 integration in the cockpit is illustrated. A pair of such sensors102, 103, 104 may be provided at the left and right of an instrumentcluster display 120 at a distance of y=+/−15 cm from the middle of thecluster 120. In further examples, a pair of such sensors 102, 103, 104may be provided at a distance from the middle of the cluster 120 of atleast 5 cm, at least 10 cm, at least 15 cm, at least 20 cm, at least 25cm, at most 5 cm, at most 10 cm, at most 15 cm, at most 20 cm, at most25 cm, at most 30 cm, about 5 cm, about 10 cm, about 15 cm, about 20 cm,about 25 cm, and about 30 cm. Each sensor 102, 103, 104 may have a coneopening of about 30°. In another example, the cone opening of eachsensor 102, 103, 104 may be at least 10°, at least 20°, at least 30°, atleast 40°, at least 50°, at least 60°, at least 70°, at least 80°, atleast 90°, at least 100°, at least 110°, at least 120°, at least 130°,at least 140°, at least 150°, at most 10°, at most 20°, at most 30°, atmost 40°, at most 50°, at most 60°, at most 70°, at most 80°, at most90°, at most 100°, at most 110°, at most 120°, at most 130°, at most140°, at most 150°, about 10°, about 20°, about 30°, about 40°, about50°, about 60°, about 70°, about 80°, about 90°, about 100°, about 110°,about 120°, about 130°, about 140°, or about 150°. The cones may bedirected in a way such that they are capable of always detecting thehead of the driver 100 and with no obstructions by the instrumentcluster 120 or the steering wheel. By using a pair of sensors 102, 103,104 arranged accordingly, all sizes of drivers 100 and also allreasonable head movements can be traced. It is estimated that a range of110° is the movement of the drivers head and body; however, a range ofother movements may be detected depending on the range of the sensors102, 103, 104 used as described throughout this disclosure. As such, thedriver 100 can be traced when he looks into exterior mirrors or CameraMonitor System (hereinafter “CMS”) displays. The typical distance fromthe eyes to the sensors may be about 75 cm.

FIG. 2 is a diagram illustrating the arrangement of an upper ToF sensor105 according to an example of the present disclosure. FIG. 3 is adiagram illustrating the arrangement of a center console ToF sensor 106according to an example of the present disclosure.

In this example, there is a ToF sensor 105 in the roof console or at thetransition between the windscreen and roof 112 on the driver's 100 sidefor determining the head position of the driver 100. In addition,another ToF sensor 106 may be located in the center console for HumanMachine Interface (hereinafter “HMI”) or other applications.

The ToF sensor 105 may be used to detect the position of the occupant100. Mainly the head position is detected, as well as the body. Thisinformation may be used to adjust the Head Up Display (hereinafter“HUD”) according to the head position since objects displayed in the HUDmust be moved according to the head position. For example, if aparticular location is to be marked with a symbol in the HUD, thelocation of the head should be known accurately.

The sensor 106 in the center console may detect gestures and approachand may be used for “swiping” the display contents from the HMI/Displayof the Center Console to the Dashboard displays. In addition, whenapproaching the display, a user 100 may see the position of his or herhand with respect to the display content. The displayed hand can be usedto enlarge symbols or fields that may otherwise be missed. The displayedhand combines gesture control with a touch display. According to thisexample, the display may then be divided into quadrants and therespective quadrant can then be zoomed in or zoomed out.

Still referring to FIG. 2, an example arrangement of the TOF(time-of-flight) sensor 105 in the roof liner 112 is illustrated, andwhich may be used for head and body tracking. As shown, the sensor 105may be arranged in a way that the head and upper body for drivers of allsizes can be detected, and the sensor 105 may include margins forsmaller users 100 (who may sit closer to the front) and larger users100. In one example, the sensor 105 has a cone of 90° and operates at arange of about 40 cm. In further examples, the cone opening of thesensor 105 may be at least 20°, at least 30°, at least 40°, at least50°, at least 60°, at least 70°, at least 80°, at least 90°, at least100°, at least 110°, at least 120°, at least 130°, at least 140°, atleast 150°, at least 160°, at least 170°, at most 20°, at most 30°, atmost 40°, at most 50°, at most 60°, at most 70°, at most 80°, at most90°, at most 100°, at most 110°, at most 120°, at most 130°, at most140°, at most 150°, at most 160°, at most 170°, about 20°, about 30°,about 40°, about 50°, about 60°, about 70°, about 80°, about 90°, about100°, about 110°, about 120°, about 130°, about 140°, about 150°, about160°, or about 170°. The sensor 105 may operate at a range of at least 5cm, at least 10 cm, at least 15 cm, at least 20 cm, at least 25 cm, atleast 30 cm, at least 35 cm, at least 40 cm, at least 45 cm, at least 50cm, at least 55 cm, at least 60 cm, at least 65 cm, at least 70 cm, atleast 75 cm, at least 80 cm, at most 5 cm, at most 10 cm, at most 15 cm,at most 20 cm, at most 25 cm, at most 30 cm, at most 35 cm, at most 40cm, at most 45 cm, at most 50 cm, at most 55 cm, at most 60 cm, at most65 cm, at most 70 cm, at most 75 cm, at most 80 cm, about 5 cm, about 10cm, about 15 cm, about 20 cm, about 25 cm, about 30 cm, about 35 cm,about 40 cm, about 45 cm, about 50 cm, about 55 cm, about 60 cm, about65 cm, about 70 cm, about 75 cm, or about 80 cm.

Referring to FIG. 3, the arrangement of the TOF (time-of-flight) sensor106 in the center console is illustrated, and may be used as both aproximity sensor to detect and prepare operating on the touch screen andas a gesture sensor for the operation of the infotainment system(gestures first and palm mainly). As shown, the sensor 106 may bearranged in a way that the hand, both of driver 100 or passenger, can bedetected with maximum comfort, and the sensor 106 may include marginsfor smaller users 100 (sitting closer to the front) and larger users 100(who typically have longer arms). In one example, the sensor 106 has acone of 90° and operates at a close range of about 5 cm. In furtherexamples, the cone opening of the sensor 106 may be at least 20°, atleast 30°, at least 40°, at least 50°, at least 60°, at least 70°, atleast 80°, at least 90°, at least 100°, at least 110°, at least 120°, atleast 130°, at least 140°, at least 150°, at least 160°, at least 170°,at most 20°, at most 30°, at most 40°, at most 50°, at most 60°, at most70°, at most 80°, at most 90°, at most 100°, at most 110°, at most 120°,at most 130°, at most 140°, at most 150°, at most 160°, at most 170°,about 20°, about 30°, about 40°, about 50°, about 60°, about 70°, about80°, about 90°, about 100°, about 110°, about 120°, about 130°, about140°, about 150°, about 160°, or about 170°. The sensor 105 may operateat a range of at least 5 cm, at least 10 cm, at least 15 cm, at least 20cm, at least 25 cm, at least 30 cm, at least 35 cm, at least 40 cm, atleast 45 cm, at least 50 cm, at least 55 cm, at least 60 cm, at least 65cm, at least 70 cm, at least 75 cm, at least 80 cm, at most 5 cm, atmost 10 cm, at most 15 cm, at most 20 cm, at most 25 cm, at most 30 cm,at most 35 cm, at most 40 cm, at most 45 cm, at most 50 cm, at most 55cm, at most 60 cm, at most 65 cm, at most 70 cm, at most 75 cm, at most80 cm, about 5 cm, about 10 cm, about 15 cm, about 20 cm, about 25 cm,about 30 cm, about 35 cm, about 40 cm, about 45 cm, about 50 cm, about55 cm, about 60 cm, about 65 cm, about 70 cm, about 75 cm, or about 80cm.

Further aspects, for example, include moving content on a display, suchas a head-up display in particular on windshield, depending on head, eyeand/or body position and/or on field of view. Content may be adapted inrelation to the vehicle car surroundings by taking at least threepositions into account—eyes, the image being displayed, and the actualobject.

In an example, to compensate for a relative movement between eyes andimage on display and/or actual object, the position of an icon on adisplay may be adapted. There may be a fixed relationship between theposition of the eyes and the display such that the icon moves on thedisplay to follow an object. In another example, the eyes may moverelative to the display and the icon may follow this movement, i.e. theicon aligns itself relative to eyes. In a further example, a warningicon moves into the field of view of the eyes due to movement of thevehicle relative to the object.

Other aspects may include attracting the attention of a driver 100 toensure awareness. In order to attract the attention of a driver 100, alight, a vibration and/or a sound effect may be triggered to alert of adangerous circumstance. The light effects can be in the field of view ofthe driver 100, provided by light modules already installed within carlike door illumination, dashboard illumination, window illumination, andthe like. Vibration can be generated by the seat, the steering wheel oran element in direct contact with the driver 100. The effect may bepreselected by the driver 100, or automatically selected based ondetected conditions and/or a detected status of the diver 100.

In addition, sensor arrangements may provide additional controlfeatures. In an example, selection of a menu function, changing betweenmenus, and other user interface functions may be performed via eye, headand/or body movement detection. In one example, recognizing the identityor condition of a driver 100 results in automatic vehicle configurationssuch as seat position, steering wheel position, language, address book,favorites, navigation places, limiting of velocity, blocking selected orall function. Recognition may be achieved via pupil identification, faceidentification, and may including detection of a condition of the driver100 such as mood, tiredness, and drug or alcohol influence.

The sensors 102, 103, 104, 105, 106 described above in reference toFIGS. 1-3 may have a number of different functions and applications.Some of the functions and application are described in Table 1 below, asan example of the functions which each or all of the sensors 102, 103,104, 105, 106 and sensor arrangements may perform or applications forwhich they are used. These functions and applications are only anexample, and the present disclosure is not limited to these functionsand applications. In Table 1, the ++ symbol indicates that the sensor102, 103, 104, 105, 106 is the most preferred sensor 102, 103, 104, 105,106 for performing the example function, the + symbol indicates that thesensor 102, 103, 104, 105, 106 is a preferred sensor 102, 103, 104, 105,106 for performing the example function, the o symbol indicates that thesensor 102, 103, 104, 105, 106 is a neutral sensor 102, 103, 104, 105,106 for performing the example function, and the − symbol indicates thatthe sensor 102, 103, 104, 105, 106 is a less preferred sensor 102, 103,104, 105, 106 for performing the example function.

TABLE 1 Example Relevance of Sensor Arrangements Sensors 102, 103,Sensor Sensor 104 105 106 illustrated illustrated illustrated Examplefunction in FIG. 1 in FIG. 2 in FIG. 3 Occupant Monitoring + ++ − Driveridentification Drowsiness & Fatigue detection Distraction detection Headorientation Eye gaze tracking Facial expression & analysis Number ofoccupants Occupant classification (gender, age, body type, etc.) GestureRecognition − o ++ Hand gesture of driver and passenger (left-, righthand) Gestures thumb up open palm fist fist action grabbing of objectreleasing of object proximity to an object proximity to a sensor ObjectRecognition/Classification o + ++ Type of object: Bottle, smart phone,spectacles, keys Classification of object: size, orientation, position

FIG. 19 is a diagram illustrating an exemplary system 200 that canmonitor or detect user movements or the environment for controlling anumber of vehicle components and conditions.

Referring to FIG. 19, the system 200 includes one or more sensorarrangements 210 which can be the same or similar to the sensorarrangements and sensors 102, 103, 104, 105, 106 described in referenceto FIGS. 1-3. The system 200 further includes an interior controller orelectronic control unit (ECU) 220 in communication with the sensorarrangements 210 and one or more vehicle components. The ECU 220 may beconfigured to control each of the one or more vehicle components. Thevehicle components may include, for example, an air vent 231, dashboardlighting 232, switches 233, a smartphone 234, a cup holder 235, amorphing surface 236, a door pocket 237, a door armrest 238, a centerconsole 239, a trunk 240, a seat 241, a seat back 242, and a roof 243.Each of these components may move in response to one or more programedmotions or environments being detected. As a result, a system 200 isprovided that can monitor all kind of movements and especially movementof objects in the interior of the vehicle. For example, the system 200may anticipate and trace movement of objects in the vehicle interior,such as bottles, cups, keys, glasses, phones, and smart phones. Toachieve these functions, the system may include one or moreelectro-mechanical actuators, an arrangement of sensors 210, control bythe ECU 220, and the software/algorithms for programming the ECU 220.

Materials, actuators, and structures used for forming a morphing surfacesuch as morphing surface 236 are known to a person of ordinary skill inthe art, for example, as described in U.S. Pat. Nos. 7,858,891 B2 and10,173,615 B2.

FIG. 20 is a flow chart illustrating an exemplary process of controllinga morphing surface 236 using a system 200 with one or more sensorarrangements 210 and an ECU 220.

Referring to the example illustrated in FIG. 20, the process may beinitiated at first step 245 after a start step in which the system 200initiates a self-test to determine whether the sensors 102, 103, 104,105, 106, ECU 220, and all components are operating properly. At step245, if any malfunctions are detected, the system 200 may indicate thedetected malfunctions to the user 100 or may initialize a programmedmalfunction protocol to self-cure the one or more detected defects. Inthis example, the next step 250 after the self-test is to read the inputof TOF sensor 105 in the roof 112 of the vehicle, as described inreference to FIG. 2. In step 260, the sensor may detect whether an armmovement is made, and the ECU 220 may continue to read the TOF sensor105 if no arm movement is detected. In response to an arm movement beingdetected, in step 265, the ECU 220 may read one or more of the othersensors 102, 103, 104, 106 to determine whether a recognized gesture ormovement is made. If no recognized gesture or movement is determined tobe made, the ECU 220 may restart the detection process by returning backto step 250. If a gesture or movement is recognized, the object anddirection affiliated with the gesture and movement may be determined instep 270. Once the object and direction are determined in step 270, anaccompanying actuation in response to the determined object anddirection may be calculated in step 275, and the ECU 220 may send asignal to one or more of the vehicle components 232, 233, 234, 235, 236,237, 238, 239, 240, 241, 242, 243, such as the morphing surface 236, forinitiating the actuation mechanism.

One practical example of the above process being implemented may be forcontrolling the morphing surface 236 to accommodate a bottle of a driver100. In this example, a driver 100 may grab a bottle out of his pocketor bag, and the TOF sensor 105 may detect arm movement. The ECU 220 maydetermine that the bottle does not approach the cup holder, and detectthe bottle in the driver's hand. The one or more sensors 102, 103, 104,105, 106 may estimate the size of the bottle for classificationpurposes. If the driver 100 opens the bottle cap, the one or moresensors 102, 103, 104, 105, 106 may monitor the movement of the hand andarm of the driver 100, and the ECU 220 may determine that this movementis not relevant for an actuation mechanism to be initiated. However, theclassification of the bottle may change from closed bottle to openbottle. If the driver 100 drinks from the bottle, the ECU 220 may againdetermine that this is not relevant for an actuation mechanism. If thedriver 100 closes the bottle and moves the bottle towards the cupholder, the one or more sensors 102, 103, 104, 105, 106 may detect themovement, and the ECU 220 may determine the movement is a relevantmovement and calculate the position of the movement. The ECU 220 maycalculate the expected position on the morphing surface 236 and the sizeof the bottle, and actuate the forming of a hole in the morphing surface236 based on the calculated expected position and the size of thebottle. The sensors 102, 103, 104, 105, 106 may detect placement of thebottle—at which point movement has stopped, and the ECU 220 may initiatea command to lock the bottle and send the command to the morphingsurface 236 actuators. At this point, the driver 100 may release his orher grip on the bottle, and the sensors 102, 103, 104, 105, 106 maydetect the released hand. The ECU 220 may store the position andclassification of the bottle including the consumed volume and type ofdrink, and may forward this information to an Internet of Things (IoT)environment.

The above described example is only one practical example of the processillustrated in FIG. 20. Other examples may involve objects other than abottle such as car keys, home keys, glasses, smart phones, among otherobjects. Each of these objects may also be registered, their whereaboutsmay be visible remotely in the IoT environment, and their position maybe used to improve the performance of other devices. For example, withsmart phones, the exact position as determined may be used to improvethe performance of wireless charging devices.

Uniform Overlay Surface for Sensor and Display.

FIG. 4 is a cross-sectional view of an exemplary fastening structure.Referring to FIG. 4, a seamless surface of a dashboard includes astructure of a partially translucent layer 10, which is designed as adecorative cover layer, and a second carrier layer 20. The partiallytranslucent layer 10 and the second carrier layer 20 may be producedusing multi-component injection molding.

The topcoat may be decorated by various coating methods such asvarnishing, PVD, IML, IMD, PUR flooding, and others. In addition,paints, which do not serve for appearance, but are applied forenvironmental influences such as resistance to scratching or imprintsapplied. A self-healing coating can also be applied. Another exampleincludes an anti-reflective surface to avoid unwanted reflections.

Still referring to FIG. 4, the carrier layer may have recesses 30 incertain areas. These recesses 30 may be used to position sensors and/ordisplays. For this purpose, fastening devices 40 may be present. In thisexample, these fastening devices 40 are designed as clips but they arenot limited thereto. The fasteners 40 may be attached to the supportlayer 20 so that no bumps on the top layer arise and thus thehigh-quality appearance is not disturbed.

Sensors which can be installed include but are not limited to ToFsensors, cameras, IR sensors, radar, ultrasound, capacitive sensors,brightness sensors, LIDAR sensors, among others.

In a further example, a plurality of frameless displays can be mounted.These displays may be technically and operationally linked with oneanother via software so that they can appear as a single, wide screenwhen viewed from the outside.

Integration of Holographic Displays in Bionic Structure or Dashboard.

FIGS. 5A and 5B are front perspective views of a display device and aseamless dashboard.

The recent trend in automotive displays has been the use of Heads-UpDisplay that allow the projection of information in front of the driverand align the projected images and objects to reality. The projectedmedia can be a combined screen, the car windshield or a hologram that isprojected in front of the car. In all these examples, a calibration forthe display device is typically needed so that the alignment can be doneprecisely. It is easy to do this alignment with precise measuringequipment during research and development but not in mass production.

A calibration method is described below that includes steps that can beperformed in mass production. For example, these steps can be performedon an assembly line such as an assembly line of the module supplier(tier 1) of the dashboard integrating the display device, or the a carassembly line where all the components including the display device, thewindscreen, the car body and the dashboard are joined. The disclosedmethod may, for example, be used to anticipate the tolerances from thecar body, the dashboard, the windscreen mounting, and to provide meansto measure and compensate the deviations in an efficient way.

Referring to FIG. 5A, an outer frame for the display device 300 isillustrated. The outer frame may be fixed to the dashboard 310 and thusto the car body which contains 1 to 6 gears 315 (mechanically orelectrically driven) that can compensate in the x/y/z direction andangularly for Yaw/Pitch/Roll. The display 300 within its frame may bepre-assembled into the dashboard 310 (e.g. at the module supplier) andthen lifted into the car body and fixed (e.g. by a robot). The car bodymay then be fixed on a test stand and the display pointed towards a testpattern. Either human operator or test camera may measure the alignmentof the projected object and a correction value may be calculated toapply to the above mentioned gears. Either a manual operator or anautomated drive may move the gears 315 until the best alignment isgiven. Then the gear 315 may be blocked with a locking device. Analignment of the image compared with a test chart may be displayed asshown in 320 for calculating the correction values and gear rotation.Referring to FIG. 5B, a front perspective view of an exemplary seamlessdashboard 315 is illustrated.

Full Windshield Head Up Display.

FIG. 6 is a diagram illustrating an exemplary HUD projector 410. FIG. 7is a rear perspective view of an exemplary projector model of picoprojectors 410. FIG. 8 is a front perspective view of an exemplaryprojector model of pico projectors 410.

Referring to FIGS. 6-8, instead of one large display being mirrored inthe windshield 400, at least two simple projectors 410 may be used toproject images anywhere on the windshield 400. The projection of twoprojectors 410 may be processed to provide one rectified projection onthe windshield 400. Keystone corrections may be performed.

The windshield 400 may be provided with a foil to enhance image quality,either applied on the inside or between two glasses. The foil may bearranged so that it can reflect specific wavelengths. The foil may beplaced on the windshield 400 by gluing or by adhesion, and can also beplaced between the windshield 400 to substitute the typically used foil,which is used for safety aspects.

In an example, advertisements may be displayed on the windshield 400when the car is not driving. Pedestrians or other road users may also beable to see the advertisements on the display which is used as a screen.

In one example, the brightness is the maximum allowed brightness allowedby the projector 410. Luminance may be laser class 1 and the minimalcontrast to achieve may be 1:5000 for the projector 410. The brightnessof the projector 410 may be 0.01 mW. Luminance may be 100 Candela.

Sharpness as the windshield 400 is curved may need to be taken intoconsideration. In typical prior applications, stitching has not beenimplemented. In this example, if there are a pair of projectors 410,stitching in the middle of the two pictures may be applied. Theadvantage is to provide a bigger area for projection. With that, aseamless presentation of the display with uniform luminance may beachieved. A single calibration may be done for installing the systemwhere the projectors 410 are aligned with the shape of the windshield400.

FIG. 9 is a front perspective view of an exemplary attachment of a picoprojector 410 in or at an A-pillar 420.

Referring to FIG. 9, the position of the projector 410 is in/at theA-pillar 420. The projector 410 as such is adjustable, to adjust thepicture. To adjust the projectors 410, a user may also use actuators anda calibration system. The mechanical attachment as such is not defined.There can also be a service flap or a similar structure integrated inthe A-Pillar 420 to re-adjust or exchange parts. This flap may also actas a decorative trim part that is translucent and has a nice appearance.The flap can protrude out of the A-pillar 420 cover. In one example, theflap is optically clear so that there is no loss in brightness. Inanother example, there is an open space where the projector 410 islocated. The projectors 410 may be mounted so that they don't collide orinterfere with safety installations, like airbags or pre-crashapplications. In an example, a cone may extend from the lens in thedirection of the laser being applied to the windshield 400, and theprojector 410 may be embedded within the A-pillar 420. The cone mayclose the gap or opening formed in the A-pillar 420. For example, anumber of different cones may be used. The cone may be a flexible conethat can be moved to close the hole in the direction of the lasers. Inone example, the distance from the lens of the projector 410 to thewindshield 420 may vary from 1 mm to 150 mm. The projector may beattached to the A-pillar 420, to the body of the car, or to anotherstructural feature or carrier which is attached to the body of the car.

Transparent Dashboard with Integrated Light Guide.

In recent trends in the automotive industry, more and more product partsof the vehicle dashboard are made of plastic. At the same time, morelighting systems and light guides are used in the interior of the car.Also, new developments allow the possibility of making plastic partsusing advanced technologies such as bionic structures, 3D-printing, etc.. . . . However, separate lighting units/light guides are still used andattached to the main structure for lighting.

FIG. 21 is a top perspective view of an exemplary light guide 500 andmanufacturing methods thereof. According to an example, the lightingdevices and light guides 500 may become a part of the dashboard plasticstructure 510. In other words, the lighting devices and light guides 500may be molded or printed with transparent (clear or colored) materialand used as light guides 500. The injection of light may work the sameway as in the case of separate light guides 500. The emission of thelight can be modulated, if necessary, with screen printing methods.

Moving Touch Pad to Follow Passenger Position.

As autonomous vehicles become more prevalent, vehicle drivers andpassengers may desire to select and change positions more freely. As thefocus of the driver on the pure driver function can be relaxed, thestrict requirements for passive safety (airbags, seat belts) can also bepartly relaxed and replaced by better active safety. Self-drivingvehicles according to SAE Levels 3 and 4 may allow drivers to recline toa more relaxed position (for level 3), similar to Aircraft seats, andpotentially to a totally reclining position (for level 4). As a result,the established ergonomics and operation of car systems such asinfotainment, car settings, climate control and others may no longer beaccessible or work properly. Inaccessibility of the car systems resultin the driver/passenger not being able to easily reach the controlswhich are usually mounted in the dashboard or the center console of aconventional car.

Accordingly, an exemplary autonomous driving vehicle may include amovable center console and a touch pad mounted on the center console atan adaptable angle. The center console may have several degrees offreedom (e.g. movable in the x-direction and the z-direction, androtatable about the z-axis). The touch pad may be tuned around severalaxes (for example, the y-axis and the z-axis). As such, the touch padmay be properly seen and reached by the driver/passenger in at least 3positions: normal driving position, relaxed driving position, andsleeping/lying position. Sensors in the car and in the car drive traincan determine the status of car and driver, and based on the determinedstatus, the sensors may send signals directing a change in the positionto a maximum comfort level.

Display Cooling with Ventilation Elements.

As cars continue to have more and more additional monitors in adashboard, climate management for the electronic components of thedisplays is already becoming a problem. Climate management is more andmore important as the number of displays increases because theoverheating of displays may also affect the vehicle occupants' climate.

Modifications to air conditioner openings, and the elimination of heataccumulation are described. In an example, a ventilation system includesan air duct system positioned within the dashboard, which receives itsair from the current air conditioning system before the air duct systempasses through a directional distributor at the interior of the vehicle.For this purpose, a defined opening is directed into a designatedchannel structure and guided close to the back of the monitor. Here, theair warms and thus the air can continue to pull up/back. Therefore, thisarrangement allows the cooling systems to be positioned below themonitors so that the air is led up past the monitors. The air supply infront of the distributor should not be switched off. The lockingmechanisms, for example via actuators, may lie between the distributorsystem and lamellae exit. Accordingly, a ventilation system with anair-channel structure inside is provided.

FIG. 10 is a perspective view of an exemplary cooling system 600.Referring to FIG. 10, an exemplary image of a cooling system 600 with aview of the shutter 610 and lamella 620 is illustrated. Here, the air isdisconnected, and passed through air ducts, which can blow the air onthe back walls of the monitors.

FIG. 11 is a diagram illustrating an exemplary air guiding system 630.Referring to FIG. 11, the air is guided through an air vent 650 and intoan air duct 660 past the rear walls of the monitor 640, and executedbefore the closure. FIG. 12 is a diagram illustrating an example of airflowing towards the monitor 640 and to the vent. Referring to FIG. 12, aplan view is illustrated of how to cool several monitors 640 in parallelwith air channels including an outlet channel 670 and cooling channel680, in parallel. The vehicle airbag 685 and windshield 690 areillustrated. A shutter fold 675 is disposed in the outlet channel 670.

Touchskin Keys.

FIG. 13 is a side cross-sectional view of an exemplary armrest section700. FIG. 14 is a cross-sectional view of an exemplary basic structureof a touchskin configuration 710. FIG. 15 is a flow chart illustratingan exemplary operable surface at different stages. FIG. 16 is a flowchart illustrating another exemplary operable surface at differentstages.

Referring to FIGS. 13-16, rather than having a pressure bar directlyvisible in a vehicle, a pressure bar according to the illustratedexample may be integrated laterally in the armrest 700 and covered withleather. The key functions are known in response to the operatortouching the surface as in the displayed flow charts, becauseinformation about each key function may appear on a display.Furthermore, in certain examples, a preset bar combining both capacitivesensor and push button may be provided. As a result, push buttons may bereplaced with piezo actuators, creating haptic feedback withoutmovement.

As illustrated in FIG. 13, the armrest 700 includes a touchskinconfiguration 710. The light guide 712, air channel 714, and door trimcarrier 716 are illustrated with respect to the armrest 700. On thesurface of the touchskin configuration 710, screen-printed heating 718is provided with side air flow 720. As illustrated in FIG. 14, thetouchskin configuration 710 includes a decorative skin 722, a softcomponent which may be foam 724, a carrier 726, a push button 728, and amorphing sensor 730.

FIGS. 22 and 23 are perspective views of an exemplary vehicle door 731including touchskin keys 732.

Referring to FIGS. 22 and 23, the touchskin keys 732 may be hidden orslightly visible in plain sight by a user. The keys 732 may be coveredby a leather, a polyurethane, or other elastic material 734. The keys732 may be arranged in a number of different configurations. Accordingto the example below, eight keys 732 are arranged with four keys 732 intwo rows. Each key 732 may perform a different function when selected,and each key 732 may perform more the same function or more than onefunction depending on whether it is touched or pressed down. As per themagnified image below, the keys 732 may be slightly visible in plainsight. The keys 732 may be lit up at all times or may be unlit. The keys732 may only light up when a user approaches the key 732 or when a usertouches a key 732. In this example, the keys 732 are arranged at theside of the armrest, as illustrated. The keys 732 may also be arrangedin other positions at the door or dashboard.

Referring to FIG. 23, the vehicle door 731 may include one or more smartmaterials. The locations and uses described in reference to FIG. 23 areprovided only as an example. The vehicle door 731 may include a display736 for displaying door settings, a lighting 735 for function activationor deactivation, a touchskin keys 732 configuration, a diffuse side airventing 739, a printed surface heating 738, color change surfaces 737depending on ambient lighting or air temperature, and a capacitivesurface 739 for changing light color, among other functions.

See Through Vehicle Display.

FIG. 17 is a block diagram illustrating an exemplary system for asee-through display for parking purposes. During parking situations andlow speed maneuvers, the driver may take several personal precautions toobtain spatial awareness especially for the rear and front view;however, spatial awareness assist for side view may be unavailable.Unavailability of spatial awareness assist may results in scratching ofthe rim, wheel, and body of the vehicle due to invisible objects whichmay be hidden to the driver's sight and behind the door.

Referring to FIG. 17, a vehicle may be provided with one or more camerasensors on both sides thereof (for example, placed in the same pod of aCMS, camera with wide viewing angle face down). The one or more sensorsmay capture image information on both sides of the vehicle and processsuch information. For example, the sensor may give information on thedistance between body/tire to outside objects. The display on the doormay give intuitive input to the driver regarding vehicle spatialinformation.

FIGS. 24 and 25 are perspective views of an exemplary see-throughdisplay 800. Referring to FIGS. 24 and 25, the camera sensor may bepositioned on both sides of the car at the doors. A display 800 may actas a see-through display 800 which displays what would be seen on theother side of the vehicle, as though the display 800 were a clear windowor opening. That is, a camera may capture the image on the other side ofthe vehicle and display that image to the user as though he were lookingthrough a clear window when looking at the display 800. In an example,the display 800 may be positioned at the door as illustrated in FIG. 24.In another example, the camera may be positioned at the front of thevehicle with the see-through display 800 being on the front dashboard sothat a user may see through the front of the car where it is usuallyblocked by the dashboard.

The display 800 may illustrate a number of different values or warnings.For example, the display 800 may give visual information 810 on distanceto surrounding objects. The display 800 may provide warning such assounds 820 or flashing lights when an object is within a thresholddistance. The display 800 may provide different warnings sounds 820 orlights depending on a size or type of the object detected. The display800 may provide different warning sounds 820 or lights depending onwhether the object is moving or not moving, and whether approaching thevehicle or moving away from the vehicle. In an example, the see-throughdisplay 800 may blend in with the interior of the vehicle and notoperate the see-through function until it is activated by a user. Inanother example, the see-through function is automatically operated whenthe vehicle is in a specific mode such as a parking mode, or when thecar is in reverse.

It should be appreciated that the see-through display 800 significantlyenhances the driver experience by allowing a user to see a positionwhich is typically impossible to see. For example, in a larger vehiclessuch as pickup trucks, the position that a door occupies is very largeand would not allow a user to see a short object or person such as achild standing beside the door as compared to a vehicle which is low tothe ground and provides a significantly larger view to the side of thevehicle. In addition, such see-through systems may significantly easecompliance with regulations of certain countries, such as Japan, wherevehicles are required to be able to view pilots of certain heightsbesides the vehicle in order to pass compliance. While the see-throughdisplay 800 is described in reference to autonomous vehicles, it shouldbe appreciated that this feature, as well as all features of thisdisclosure, are also described in reference to non-autonomous vehiclesand this disclosure covers all vehicles

Dashboard with Rearranged Camera Monitoring Systems.

FIG. 18 is a front perspective view of an exemplary dashboard for anautonomous driving vehicle. As advancements are made in vehicle rearview capturing technology, cars may or may not have rear view mirrors.Mirrors may be replaced by so called “camera monitor systems” (CMS). Inan example, CMS systems include rear view cameras that record the rear,lateral area of the car surroundings. The captured images or videos maybe displayed in dedicated screens 1, 2. The locations and types of thesescreens are regulated by laws which may differ depending on thejurisdiction.

Referring to FIG. 18, the displayed rear view image provided by the oneor more CMS systems is or may be duplicated on other screens 3, 4, 5,whereby the legally regulated areas for displaying of the rear view arenot affected. In other words, images may be stitched from screen 1 and 2together and displayed also on the central display 3, the wide screen ofthe dashboard 4, the door screens 5, or any other display in the car.

As a result of the arrangement of the displays 1, 2, 3, 4, 5, otherpassengers are also able to see the displayed image. Further, contentcan be projected onto the image that may not comply with legalrequirements. For example, augmented reality applications for warning orother informational purposes may be projected onto the image.

In addition, such displayed images may also be used to adjust the FOV ofthe simulated mirror. In this example, the driver transfers the imagefrom 1 to 3, adjusts the FOV with touch or gesture control on the HMI,and then sends the image back to screen 1.

FIG. 26 is a front perspective view of an exemplary rear view display900. Referring to FIG. 26, the rear view display 900 is in a position ofa convention interior rearview mirror, and includes CMS informationcaptured on the left side of a vehicle and CMS information captured onthe right side of a vehicle which is combined and stitched together. Theinformation captured from the left side is displayed on the left side910 of the display 900, and the information captured from the right sideis displayed on the right side 920 of the display 900. Such displays arewell known to a person having ordinary skill in the art. For example,the arrangement and algorithms used with a display such as display 900are described in German Patent Application Publication No. 10 2014213536 A1 and counterpart U.S. Patent Application Publication No.2017/116710 A1.

However, in certain jurisdictions, the ISO standard requires thatinformation captured from the left side of a vehicle must be displayedon a display to the left of the driver and information captured from theright side of a vehicle must be displayed on a display to the right ofthe driver. In particular, the ISO standard includes rules on thelocation of CMS information to the left of the driver for informationcaptured on the left side and to the right of the driver for informationcaptured from the right side. The reason is that drivers tend to look tothe particular side that they sense a danger exists, for example, basedon the noise of another vehicle from that side. Thus, a display shouldbe on the same side as the side where the image is being captured. Thisarrangement is mandatory in a number of jurisdictions including Europe,Japan, and Korea.

To meet regulatory requirements while providing a conveniently displayedand combined, stitched image, a system and algorithm may be used fordisplaying a stitched image (hereinafter “a comfort display”) orseparate images on a display to the left and right of a vehicle occupant(hereinafter “an ISO-compliant display”).

Referring back to FIG. 18, a CMS system may include an electroniccontrol unit (ECU) 220 in communication with screens or displays 1, 2,3, 4, 5, a left side image capturing device 6 such as a left side cameraor sensor, a right side image capturing device 6 such as a right sidecamera or sensor, and other vehicle units for receiving inputs from thevehicle on vehicle speed, number of lanes on the driving road, currentdriving lane, time of driving, blind spot detection, among other inputs.The ECU 220 may also be in communication with a display 900 which iscapable of displaying a comfort display. Further, any one or more of thedisplays 1, 2, 3, 4, 5, 900 may be capable of displaying a comfortdisplay or a left or right part of an ISO-compliant display. In thisexample, the ECU 220 may be the same ECU 220 which is also incommunication with the sensor arrangements 210 or may be a different ECU200 which is not in communication with the sensor arrangements 210.

A CMS system, as described, may provide the required mirror/CMSinformation in the most convenient way, maintain improved safetyfeatures by combining the information from the vehicle, the ADAS systemand the CMS, switch to safe and ISO-compliant information when required,and free the displays 1, 2, 3, 4, 5, 900 for other purposes when theyare not required for the CMS.

FIG. 27 is a flow chart illustrating an exemplary process for displayingone or more images on the one or more displays 1, 2, 3, 4, 5, 900. Inthis example, the process may be initiated at first step 902 after astart step, in which the CMS system initiates a self-test to determinewhether the displays 1, 2, 3, 4, 5, 900, ECU 220, image capturingdevices 6, and all components are operating properly. At the first step902, if any malfunctions are detected, the system may indicate thedetected malfunctions to the user 100 or may initialize a programmedmalfunction protocol to self-cure the one or more detected defects. Inthis example, in the next step 903, the ECU 220 may read or receivevehicle data from one or more vehicle units to evaluate drivingconditions. For example, the ECU 220 may receive inputs from the vehicleon vehicle speed, number of lanes on the driving road, current drivinglane, time of driving, blind spot detection, among other inputs. In thenext step 904, the ECU 220 may evaluate whether it is safe to display acomfort display or whether an ISO-compliant display should be displayed.An example of the inputs which may be evaluated to determine whether acomfort display or ISO-compliant display should be displayed is providedbelow. In the next step 905, in response to determining that anISO-compliant display should be displayed, an ISO-compliant display iscalculated and displayed on one or more of the displays 1, 2, 3, 4, 5,900. In step 906, the ECU 220 may continue to evaluate inputs todetermine whether it is safe to display a comfort display or whether anISO-compliant display should be displayed. If it is determined to besafe to display a comfort display in step 906 or 904, a comfort displaymay be calculated and displayed on one or more of the displays 1, 2, 3,4, 5, 900 in step 907. After a comfort display is displayed, the processmay revert back to step 903 of receiving or reading inputs using the ECU220.

In an example, the ISO-compliant display may be displayed in a singledisplay of the displays 1, 2, 3, 4, 5, 900. That is, if a display 1, 2,3, 4, 5, 900 is large enough, the full CMS may be shown in a window oroverlay. Displaying in a single display may provide additional awarenessfor the danger situation and better catch the attention of a driver 100.Also, once the danger is determined to be no longer at issue, theISO-compliant CMS image can fade into a comfort display instead ofswitching off the display 1, 2, 3, 4, 5, 900 or returning it to itsoriginal content.

In one example, the criteria for determining whether a comfort displayshould be used, i.e. whether it is safe to use a comfort display, orwhether an ISO-compliant display should be used, is provided in Table 2.

TABLE 2 Criteria for Safe/Unsafe Driving and Comfort Display VarietyNumber Blind Brakes Speed speed of spot activation km/h In km/h Durationlanes warning >0.1 g Decision <6 x x x x x show full CMS >6 +/−10 1minute 1 off no show comfort km/h CMS >6 x x >1 off no show comfort CMSx x x x x yes show full CMS x x x x on x show full CMS

In Table 2, an x signifies that the input is not used for a particularscenario. Thus, in the first example, when the speed of the vehicle isless than 6 km/h, the full CMS is displayed. In another example, whenthe speed is greater than or equal to 6 km/h, the speed ranges+/−10 km/hfor a duration of 1 minute, there is 1 lane, the blind spot warning isoff, and there is no break activation, the comfort display is shown. Inanother example, when the speed is greater than or equal to 6 km/h andthere is more than one lane, the comfort display is shown. In furtherexamples, when the brake is activated, the full CMS is shown, or whenthe blind spot warning is on, the full CMS is shown. These examples areonly provided for explanation, and it should be appreciated that theinputs for determining whether or not it is safe to show a comfortdisplay are not limited to these examples.

It will be appreciated by those skilled in the art that changes could bemade to the embodiments described above without departing from the broadinventive concept thereof. It is understood, therefore, that the presentdisclosure disclosed herein is not limited to the particular embodimentsdisclosed, and it is intended to cover modifications within the spiritand scope of the present disclosure.

What is claimed is:
 1. A vehicle, comprising: one or more sensorsarranged on at least one of a dashboard, a roof, and a center console ofthe vehicle, or one or more image capturing devices for capturing one ormore images from a left side and a right side of the vehicle; anelectronic control unit (ECU) configured to communicate with the one ormore sensors or the one or more image capturing devices; and at leastone of a morphing surface, a windshield display, and one or moredisplays configured to be controlled by the ECU, wherein the one or moresensors are arranged under a black panel surface, the black panelsurface is comprised by a seamless surface of a dashboard which includesa structure of a partially translucent layer which is designed as adecorative cover layer, and a carrier layer having recesses in certainareas which are used to position sensors via fastening devices attachedto the carrier layer so that no bumps on the top layer arise and thusappearance is not disturbed, and wherein the ECU receives data from theone or more sensors on one or more of a position of an occupant, aposition of an object, a size of an object, or a shape of an object. 2.The vehicle according to claim 1, wherein the one or more sensorscomprise at least one of a Time of Flight (ToF) sensor, a camera, aninfrared (IR) sensor, a radar, an ultrasound, a capacitive sensor, abrightness sensor, and a LIDAR sensor.
 3. The vehicle according to claim1, wherein the one or more sensors comprises a plurality of sensors, anda first sensor of the plurality of sensors is configured to detect aposition of the occupant to activate or deactivate a second sensor or athird sensor of the plurality of sensors.
 4. The vehicle according toclaim 1, wherein, in response to a first sensor detecting that aposition of the occupant is outside a range of the second sensor andinside a range of the third sensor, the third sensor is activated, andin response to the first sensor detecting that a position of theoccupant is outside a range of the third sensor and inside a range ofthe second sensor, the second sensor is activated.
 5. The vehicleaccording to claim 4, wherein the second sensor and the third sensor arepositioned on a left side and a right side of an instrument clusterdisplay at a distance ranging from about 15 cm above to about 15 cmbelow the instrument cluster display.
 6. The vehicle according to claim1, wherein the one or more sensors are configured to detect a positionof at least one of a center of a face of the occupant, an outer edge ofa head of the occupant, a shoulder of the occupant, and a head of theoccupant with respect to a shoulder of the occupant.
 7. The vehicleaccording claim 1, wherein each of the one or more sensors has a fieldof view of at least about 30 degrees and operates at a close range of atleast about 5 cm.
 8. The vehicle according to claim 1, wherein thevehicle comprises the windshield display, and content on the windshielddisplay is configured to be moved depending on the data received fromthe one or more sensors.
 9. The vehicle according to claim 8, whereinthe content on the windshield display is configured to be moved tocompensate for movement of eyes of the occupant with respect to at leastone of an icon on the windshield display or an external object outsidethe vehicle.
 10. The vehicle according to claim 1, wherein the one ormore sensors are configured to detect one or more of an identification,a drowsiness or fatigue, a distraction, a head orientation, an eye gaze,a facial expression, a gender classification, an age classification, abody type, a quantity, a hand gesture, a thumbs up gesture, an open palmgesture, a fist or fist gesture, a grabbing of the object, a releasingof the object, a proximity, and a proximity to the object of or by theoccupant.
 11. The vehicle according to claim 1, wherein the one or moresensors are configured to detect one or more of a type of the object, asize of the object, an orientation of the object, and a position of theobject.
 12. The vehicle according to claim 1, further comprising one ormore of an air vent, dashboard lighting, switches, a smartphone, a cupholder, a door pocket, a door armrest, a center console, a trunk, aseat, a seat back, and a roof which is configured to be controlled bythe ECU.
 13. The vehicle according to claim 1, wherein the ECU isconfigured to initiate a self-test process to determine whether the oneor more sensors and the ECU are operating properly, in response todetecting a malfunction of any of the one or more sensors and the ECU,the ECU is configured to display the detected malfunction or initializea programmed malfunction protocol to self-cure the detected malfunction,in response to no malfunction being detected, the ECU is configured toread an input of a first sensor of the one or more sensors until an armmovement is detected by the first sensor, in response to an arm movementbeing detected by the first sensor, the ECU is configured to read asecond sensor of the one or more sensors to determine whether arecognized movement or gesture is performed, and in response todetermining that a recognized movement or gesture is performed, the ECUis configured to transmit a signal to one or more vehicle componentsbased on the recognized movement or gesture.